A cache is commonly used in a computer system to provide fast access to part of a dataset. Typically a cache memory is significantly faster than the main data store, often by more than an order of magnitude. Cache memories are usually quite small relative to a larger data store from which their contents are obtained. For example, a CPU may have a cache of 2 MiB used to accelerate access to 16 GiB of DRAM, or a 4 TiB hard disk may have 64 MiB of DRAM as its cache.
Because of the large disparity in the size of a typical cache and the dataset being accelerated the choice of which data to cache, and when, is critical. Equally critical is the choice of which data to evict from a cache and when such an eviction should take place.
A computer system may have several levels of cache, perhaps of differing speed and size, and also may have several types of cache. Caches may be generic and able to hold any data in the system, e.g. a processor L2 cache, and some caches may be specialised and able to only hold very specific types of data, e.g. a processors translation lookaside buffer used to hold only address translation tables.
Caches may be built from special hardware, e.g. processor L2 and TLB caches or caches may be ordinary DRAM used to accelerate access to data normally held on a slower medium, e.g. a magnetic disk.
Some caches may hold data that are expected to cycle through very quickly, e.g. a processor L2 cache and some hold data that may stay in cache for a long time, e.g. some page address translations in a TLB.
Those data frequently accessed so as to be held in a cache are often referred to as a “hot set” or “hot data”. As the set of hot data changes the data in the cache will be accessed less frequently and the data in the main store will be accessed more frequently. This can be viewed as a cooling of the temperature of the data in the cache and is a sign that some data must be evicted from the cache in order to make way for new, hotter, data to be cached.